Effectiveness of a virtual patient simulation training on improving provider engagement in suicide safer care.

BACKGROUND: Health care providers have a critical opportunity to mitigate the public health problem of suicide. Virtual patient simulations (VPS) allow providers to learn and practice evidence-based suicide prevention practices in a realistic and risk-free environment. The purpose of this study was to test whether receiving VPS training increases the likelihood that providers will engage in effective suicide safer care practices. METHODS: Behavioral health and non-behavioral health providers (N = 19) at a Federally Qualified Health Center who work with patients at risk for suicide received the VPS training on risk assessment, safety planning, and motivation to engage in treatment. Providers' electronic health records were compared 6 months pre- and post-VPS training on their engagement in suicide safer care practices of screening, assessment, safety planning, and adding suicide ideation to the problem list. RESULTS: Most behavioral health providers were already engaging in evidence-based suicide prevention care prior to the VPS training. Findings demonstrated the VPS training may impact the likelihood that non-behavioral health providers engage in suicide safer care practices. CONCLUSION: VPS training in evidence-based suicide prevention practices can optimize and elevate all health care providers' skills in suicide care regardless of role and responsibility, demonstrating the potential to directly impact patient outcomes.

Enhancing pre-employment transition services: A type 1 hybrid randomized controlled trial protocol for evaluating WorkChat: A Virtual Workday among autistic transition-age youth.

Autistic transition-age youth experience high rates of unemployment and underemployment, in part due to the social challenges they may face when having conversations in the workplace. In an effort to help enhance conversational abilities in the workplace, our collaborative team partnered to develop WorkChat: A Virtual Workday. Specifically, our team of scientists, community partners, and diversity and inclusion experts participated in a community-engaged process to develop WorkChat using iterative feedback from autistic transition-age youth and their teachers. With initial development complete, this study reports on the protocol that our collaborative team developed, reviewed, and approved to conduct a randomized controlled trial (RCT) to evaluate the real-world effectiveness and initial implementation process outcomes of WorkChat when integrated into post-secondary pre-employment transition services (Pre-ETS). Our aims are to: 1) evaluate whether services-as-usual in combination with WorkChat, compared to services-as-usual with an attention control, enhances social cognition and work-based social ability (between pre- and post-test); reduces anxiety about work-based social encounters (between pre- and post-test), and increases sustained employment by 9-month follow-up; 2) evaluate whether social cognitive ability and work-based social ability mediate the effect of WorkChat on sustained employment; and 3) conduct a multilevel, mixed-method process evaluation of WorkChat implementation.

A qualitative study of provider feedback on the feasibility and acceptability of virtual patient simulations for suicide prevention training.

Background: Healthcare organizations are often committed to preventing suicide among their patients, but they can struggle to adequately train providers and implement strategies grounded in evidence-based suicide prevention practices. Virtual patient simulations (VPS) offer the opportunity for providers at healthcare organizations and educational institutions to learn suicide prevention strategies using a realistic and risk-free environment. The purpose of this study was to gather feedback from leaders in the healthcare field regarding the feasibility and acceptability of VPS for their organizations. Methods: Participants (N=9) included administrators, managers, and educators from a variety of health care settings. They were invited to independently test the VPS and participate in a subsequent focus group to provide feedback. Participants were asked about VPS acceptability, satisfaction, potential fit within the intended context, feasibility of delivery, motivation to use, and likelihood of adoption. Responses were audio recorded and transcribed for coding and thematic analysis. Results: Themes emerged regarding perceived benefits of the VPS, considerations related to cost, barriers to implementation, and suggestions for improvement. Participants reported VPS trainings were acceptable and feasible, filling an important gap in the field especially around suicide safety planning, particularly for newer clinicians and students in training. Participants felt that this type of virtual training was particularly feasible given the recent increase in need for online trainings. Suggested improvements included the need to normalize the trial-and-error nature of the VPS for trainees prior to the start of the training, and to consider shortening the duration of the simulation due to learners not being able to bill for time while training. Conclusions: VPS may help to fill an important training need in the field of suicide prevention. The training suite may be best suited for certain settings, such as educational institutions, and most useful for populations including students and new clinicians. VPS may be particularly feasible for organizations that already utilize remote options for work and training.

Virtual interview training for autistic transition age youth: A randomized controlled feasibility and effectiveness trial.

LAY ABSTRACT: Autistic transition age youth struggle with obtaining employment, and interviewing is a critical barrier to getting a job. We adapted an efficacious virtual reality job interview intervention to meet the needs of autistic transition age youth, called the Virtual Interview Training for Transition Age Youth. This study evaluated whether Virtual Interview Training for Transition Age Youth can be feasibly delivered in high school special education settings and whether Virtual Interview Training for Transition Age Youth improves job interview skills, job interview self-efficacy, job interview anxiety, and access to employment. Forty-eight autistic transition age youth received school-based pre-employment services as usual with Virtual Interview Training for Transition Age Youth, while 23 autistic transition age youth received services as usual only. Local teachers trained and supervised autistic transition age youth using Virtual Interview Training for Transition Age Youth. Participants reported Virtual Interview Training for Transition Age Youth was highly acceptable. Participants receiving services as usual and Virtual Interview Training for Transition Age Youth, compared to participants receiving services as usual only, had better job interview skills and lower job interview anxiety as well as greater access to jobs. Overall, Virtual Interview Training for Transition Age Youth appears to be effective at teaching job interview skills that are associated with accessing competitive jobs. Moreover, youth enjoyed Virtual Interview Training for Transition Age Youth and teachers feasibly implemented the tool within special education pre-employment transition services. Future research needs to better understand how autistic transition age youth from culturally diverse backgrounds and different social, behavioral, or mental health challenges may respond to Virtual Interview Training for Transition Age Youth.

Initial testing of a computer-based simulation training module to support clinicians' acquisition of CBT skills for substance use disorder treatment.

Cognitive behavioral therapy (CBT) is one of the most common and effective treatments for substance use disorders (SUD); however, effective delivery of CBT depends on a wide variety of nuanced skills that require practice to master. We created a computer-based simulation training system to support the development of necessary skills for student trainees to be able to apply CBT effectively for clients with SUDs. CBT: Introducing Cognitive Behavioral Therapy is an interactive, role-play simulation that provides opportunities for clinician trainees to hone their skills through repeated practice and real-time feedback before application in a clinical setting. This is the first study that tests whether such a simulation improves trainee skills for the treatment of clients with SUDs. Graduate students (N = 65; social work, clinical psychology) completed standardized patient (SP) interviews, were randomized to the simulation training program or manual comparison condition (Project MATCH manual), and completed SP interviews three months post-baseline. Using general linear models, results indicated a significant time x group effect, with students assigned to the simulation training program showing greater improvement in "extensiveness" and "skillfulness" ratings across three skill categories: general agenda setting (p = .03), explaining CBT concepts (p = .007), and understanding of CBT concepts (p = .001). However, manual comparison participants showed greater improvement than simulation trainees in "assessing primary drug use" (prange = .013-.024). No changes in extensiveness or skillfulness of motivational interviewing (MI) style were observed. This pilot test of CBT: Introducing Cognitive Behavioral Therapy offers support for use of this novel technology as a potential approach to scale up CBT training for students, and perhaps clinicians, counseling people with SUDs.

Using community-engaged methods to adapt virtual reality job-interview training for transition-age youth on the autism spectrum.

BACKGROUND: Virtual Reality Job-Interview Training (VR-JIT) is an efficacious Internet-based intervention for adults with severe mental illness (SMI). Evaluations of VR-JIT have shown improved interview skill and access to employment in several cohorts of adults with SMI and with autism spectrum disorders (ASD). VR-JIT trains participants how to fill out job applications and handle job interviews through e-learning content and applied practice. Trainees receive feedback through in-the-moment nonverbal cues, critiques, and recommendations for improving performance. Our study sought to adapt VR-JIT for transition-age youth with ASD (TAY-ASD). METHODS: We recruited TAY-ASD and adult stakeholders from public and charter schools, transition programs, and community service providers. Participants provided feedback on VR-JIT to enhance its applicability to TAY-ASD. We used community-engaged methods to process and analyze data from TAY-ASD and stakeholders, presented their quantitative and qualitative responses to community and scientific advisory boards for review and recommendations, and adapted the intervention design and content. RESULTS: Our adaptations included adding diversity (gender; race/ethnicity) to the virtual hiring manager; shortening the interview by reducing response options; increasing social storytelling to enhance engagement with VR-JIT core components; adding employment opportunities more relevant to younger workers; reducing the reading level; and making the e-learning content more accessible by adding bullet points, voiceover, and imagery/video; and adding new learning goals. CONCLUSIONS: This study presents a rigorous and innovative community-engaged methodology for adapting VR-JIT to meet the needs of TAY-ASD. We review our engagement with TAY-ASD and stakeholders, and discuss the standardized coding scheme we used to adapt VR-JIT and the usefulness and limitations of employing this methodology in adapting other behavioral interventions.

Suicide risk assessment training using an online virtual patient simulation.

BACKGROUND: Improving the identification of and intervention with patients at risk for suicide requires innovative training techniques that safely and effectively teach or enhance practitioners' skills. Virtual patient simulations (VPS) can be particularly effective for this purpose because they allow for repetition in skill building as well as a safe space to practice difficult interactions with patients. The purpose of this study was to assess the feasibility and acceptability of a novel VPS that trains practitioners in suicide risk assessment, as well as to examine pre-post changes in suicide-related knowledge through a pilot of the VPS training. METHODS: Practitioners (n=20) were recruited from a Federally Qualified Health Center in the northeastern United States to test the feasibility and acceptability of a VPS suicide risk assessment training. A paired samples t-test was conducted to compare mean differences in practitioners' suicide risk assessment knowledge scores from pre- to post-training, on a scale of 0 to 10. RESULTS: The VPS was feasible to implement, with 18 of 20 participants using the VPS for an average of 21 to 95 minutes, and was acceptable to participants, with an average satisfaction rating of 5.82 out of 7. Participants' knowledge scores improved significantly by an average of 1.86 points from pre- to post-training. CONCLUSIONS: The VPS was feasible and acceptable to this sample of practitioners and significantly increased knowledge from pre- to post-training. As such, VPS holds promise as a technique to develop skills in suicide risk assessment.

Mechanism of Action for Obtaining Job Offers With Virtual Reality Job Interview Training.

OBJECTIVE: Four randomized controlled trials revealed that virtual-reality job interview training (VR-JIT) improved interviewing skills and the odds of obtaining a job offer among trainees with severe mental illness or autism spectrum disorder. This study assessed whether postintervention interviewing skills mediated the relationship between completion of virtual interviews and receiving job offers by six-month follow-up. METHODS: VR-JIT trainees (N=79) completed pre- and posttest mock interviews and a brief survey approximately six months later to assess whether they received a job offer. RESULTS: As hypothesized, analyses indicated that the number of completed virtual interviews predicted greater posttest interviewing skills (ß=.20, 95% posterior credible interval [PCI]=.08-.33), which in turn predicted trainees' obtaining a job offer (ß=.28, 95% PCI=.01-.53). CONCLUSIONS: VR-JIT may provide a mechanism of action that helps trainees with various psychiatric diagnoses obtain job offers in the community. Future research can evaluate the community-based effectiveness of this novel intervention.

Virtual Reality Job Interview Training and 6-Month Employment Outcomes for Individuals with Substance Use Disorders Seeking Employment.

BACKGROUND: Individuals with substance use disorders (SUDs) have low employment rates and job interviewing is a critical barrier to employment for them. Virtual reality training is efficacious at improving interview skills and vocational outcomes for several clinical populations. OBJECTIVE: This study evaluated the acceptability and efficacy of virtual reality job interview training (VR-JIT) at improving interview skills and vocational outcomes among individuals with SUDs via a small randomized controlled trial (n=14 VR-JIT trainees, n=11 treatment-as-usual (TAU) controls). METHODS: Trainees completed up to 10 hours of virtual interviews, while controls received services as usual. Primary outcome measures included two pre-test and two post-test video-recorded role-play interviews and vocational outcomes at six-month follow-up. RESULTS: Trainees reported that the intervention was easy-to-use and helped prepared them for future interviews. While co-varying for pre-test role-play performance, trainees had higher post-test role-play scores than controls at the trend level (p<0.10). At 6-month follow-up, trainees were more likely than controls to attain a competitive position (78.6% vs. 44.4%, p<0.05, respectively). Trainees had greater odds of attaining a competitive position by 6 month follow-up compared to controls (OR: 5.67, p<0.05). VR-JIT participation was associated with fewer weeks searching for a position (r= -0.36, p<0.05). CONCLUSIONS: There is preliminary evidence that VR-JIT is acceptable to trainees. Moreover, VR-JIT led to better vocational outcomes with trainees having greater odds of attaining a competitive position by 6-month follow-up. Future studies could evaluate the effectiveness of VR-JIT within community-based services.

Job Offers to Individuals With Severe Mental Illness After Participation in Virtual Reality Job Interview Training.

OBJECTIVE: Individuals with severe mental illness have low employment rates, and the job interview presents a critical barrier for them to obtain competitive employment. Prior randomized controlled trials (RCTs) indicated that virtual reality job interview training (VR-JIT) improved job interviewing skills among trainees. This study assessed whether VR-JIT participation was associated with greater odds of receiving job offers in the six-months after completion of training. METHODS: To assess the efficacy of VR-JIT, trainees (N=39) in the method and a comparison group (N=12) completed a brief survey approximately six months after participating in the RCTs. Primary vocational outcome measures included receiving a job offer and number of weeks searching for employment. RESULTS: A larger proportion of trainees than comparison participants received a job offer (51% versus 25%, respectively). Trainees were more likely to receive a job offer than comparison participants (odds ratio=9.64, p=.02) after analyses accounted for cognition, recency of last job, and diagnosis. Trainees had greater odds of receiving a job offer for each completed VR-JIT trial (odds ratio=1.41, p=.04), and a greater number of completed VR-JIT trials predicted fewer weeks of searching for employment (ß=-.74, p=.02). CONCLUSIONS: Results provide preliminary support that VR-JIT is a promising intervention associated with enhanced vocational outcomes among individuals with severe mental illness. Given that participants had minimal access to standardized vocational services, future research could evaluate the effectiveness of VR-JIT among individuals with and without access to standardized vocational services as well as evaluate strategies to implement VR-JIT within a large community mental health service provider.

Virtual reality job interview training and 6-month employment outcomes for individuals with schizophrenia seeking employment.

BACKGROUND: Individuals with schizophrenia have low employment rates and the job interview presents a critical barrier for them to obtain employment. Virtual reality training has demonstrated efficacy at improving interview skills and employment outcomes among multiple clinical populations. However, the effects of this training on individuals with schizophrenia are unknown. This study evaluated the efficacy of virtual reality job interview training (VR-JIT) at improving job interview skills and employment outcomes among individuals with schizophrenia in a small randomized controlled trial (n=21 VR-JIT trainees, n=11 waitlist controls). METHODS: Trainees completed up to 10h of virtual interviews using VR-JIT, while controls received services as usual. Primary outcome measures included two pre-test and two post-test video-recorded role-play interviews scored by blinded raters with expertise in human resources and self-reported interviewing self-confidence. Six-month follow-up data on employment outcomes were collected. RESULTS: Trainees reported that the intervention was easy-to-use, helpful, and prepared them for future interviews. Trainees demonstrated increased role-play scores between pre-test and post-test while controls did not (p=0.001). After accounting for neurocognition and months since prior employment, trainees had greater odds of receiving a job offer by 6month follow-up compared to controls (OR: 8.73, p=0.04) and more training was associated with fewer weeks until receiving a job offer (r=-0.63, p<0.001). CONCLUSIONS: Results suggest that VR-JIT is acceptable to trainees and may be efficacious for improving job interview skills in individuals with schizophrenia. Moreover, trainees had greater odds of receiving a job offer by 6-month follow-up. Future studies could evaluate the effectiveness of VR-JIT within community-based services.

Brief report: vocational outcomes for young adults with autism spectrum disorders at six months after virtual reality job interview training.

Young adults with high-functioning autism spectrum disorder (ASD) have low employment rates and job interviewing presents a critical barrier to employment for them. Results from a prior randomized controlled efficacy trial suggested virtual reality job interview training (VR-JIT) improved interviewing skills among trainees with ASD, but not controls with ASD. We conducted a brief survey with 23 of 26 participants from this study to evaluate their vocational outcomes at 6-month follow-up with a focus on whether or not they attained a competitive position (employment or competitive volunteering). Logistic regression indicated VR-JIT trainees had greater odds of attaining a competitive position than controls (OR 7.82, p < 0.05). Initial evidence suggests VR-JIT is a promising intervention that enhances vocational outcomes among young adults with high-functioning ASD.

Virtual Reality Job Interview Training for Veterans with Posttraumatic Stress Disorder.

BACKGROUND: Veterans with posttraumatic stress disorder (PTSD) have low employment rates and the job interview presents a critical barrier for them to obtain competitive employment. OBJECTIVE: To evaluate the acceptability and efficacy of virtual reality job interview training (VR-JIT) among veterans with PTSD via a small randomized controlled trial (n=23 VR-JIT trainees, n=10 waitlist treatment-as-usual (TAU) controls). METHODS: VR-JIT trainees completed up to 10 hours of simulated job interviews and reviewed information and tips about job interviewing, while wait-list TAU controls received services as usual. Primary outcome measures included two pre-test and two post-test video-recorded role-play interviews scored by blinded human resource experts and self-reported interviewing self-confidence. RESULTS: Trainees attended 95% of lab-based VR-JIT sessions and found the intervention easy-to-use, helpful, and prepared them for future interviews. VR-JIT trainees demonstrated significantly greater improvement on role-play interviews compared with wait-list TAU controls (p=0.04) and demonstrated a large effect for within-subject change (Cohen's d=0.76). VR-JIT performance scores increased significantly over time (R-Squared=0.76). Although VR-JIT trainees showed a moderate effect for within-subject change on self-confidence (Cohen's d=0.58), the observed difference between conditions did not reach significance (p=0.09). CONCLUSIONS: Results provide preliminary support that VR-JIT is acceptable to trainees and may be efficacious for improving job interview skills and self-confidence in veterans with PTSD.

Virtual reality job interview training for individuals with psychiatric disabilities.

Services are available to help support existing employment for individuals with psychiatric disabilities; however, there is a gap in services targeting job interview skills that can help obtain employment. We assessed the feasibility and efficacy of Virtual Reality Job Interview Training (VR-JIT) in a randomized controlled trial. Participants were randomized to VR-JIT (n = 25) or treatment-as-usual (TAU) (n = 12) groups. VR-JIT consisted of 10 hours of simulated job interviews with a virtual character and didactic online training. The participants attended 95% of laboratory-based training sessions and found VR-JIT easy to use and felt prepared for future interviews. The VR-JIT group improved their job interview role-play performance (p ≤ 0.05) and self-confidence (p ≤ 0.05) between baseline and follow-up as compared with the TAU group. VR-JIT performance scores increased over time (R = 0.65). VR-JIT demonstrated initial feasibility and efficacy at improving job interview skills and self-confidence. Future research may help clarify whether this intervention is efficacious in community-based settings.

Simulated job interview improves skills for adults with serious mental illnesses.

Adults with serious mental illnesses (e.g., Autism Spectrum Disorder [ASD], schizophrenia, post-traumatic stress disorder [PTSD]) often have difficulties obtaining employment. The Job Interview Training System with Molly Porter, developed in collaboration with Yale and Northwestern Universities and vocational rehabilitation specialists with funding from The National Institutes of Health (R43/44MH080496), allows learners to practice job interviews on computers in a stress free environment. The system includes user-driven educational materials, an interactive job application, a practice simulation with a fictional interviewer (Molly Porter), and extensive feedback. SIMmersion's PeopleSIM™ technology allows each conversation with Molly to provide a unique interview experience, enabling users to gain confidence while building skills. The on-screen coach provides insight during the conversation, and a comprehensive after-action review provides learners with feedback on the entire interview. In a randomized control trial, the system was proven effective at improving participants' interview skills and confidence. Ninety-six (96) unemployed adults with ASD (n=26), schizophrenia/other (n=37) or PTSD (n=33) were recruited. Participants were randomized into control (n=32) and experimental (n=64) conditions. The control group was "wait-listed" to receive training, and the experimental group used the training system with Molly Porter. Both groups completed pre- and post-intervention role-play interviews and self-assessment questionnaires. Analyses of covariance showed that the simulation provided a highly significant training effect, with experimental group participants scoring better in the role-play interviews and self-assessing higher than control group participants. By increasing skills and confidence, this system may ultimately reduce the length of unemployment for adults with mental illnesses.

Virtual reality job interview training in adults with autism spectrum disorder.

The feasibility and efficacy of virtual reality job interview training (VR-JIT) was assessed in a single-blinded randomized controlled trial. Adults with autism spectrum disorder were randomized to VR-JIT (n = 16) or treatment-as-usual (TAU) (n = 10) groups. VR-JIT consisted of simulated job interviews with a virtual character and didactic training. Participants attended 90 % of laboratory-based training sessions, found VR-JIT easy to use and enjoyable, and they felt prepared for future interviews. VR-JIT participants had greater improvement during live standardized job interview role-play performances than TAU participants (p = 0.046). A similar pattern was observed for self-reported self-confidence at a trend level (p = 0.060). VR-JIT simulation performance scores increased over time (R(2) = 0.83). Results indicate preliminary support for the feasibility and efficacy of VR-JIT, which can be administered using computer software or via the internet.